Multivariate CAViaR

An Insightful Approach to Risk Modeling

Abstract

This thesis builds upon previous literature for modeling value-at-risk (defined as an x% quantile of an asset’s daily returns) using non-linear ARMA terms by adding exchange-traded funds (ETFs) as explanatory variables that are combined into principal component vectors at the forecast origin. Combining these principal component vectors with transformations of lagged autoregressive response variables results in a model that produces similar predictive accuracy during periods of relatively low volatility along with more insight into the drivers of the changes in the response variable. In fact, one insight gained from the new model is a method of detecting changepoints in the economy by measuring the angle between resultant vectors calculated from the combination of principal component vectors during different time periods. This method, along with analysis of the statistical significance of the lagged ETFs, allows for insight into changes in the underlying economy.

Background and Introduction

When modeling financial time series, simply considering the mean and the variance is insufficient for an accurate depiction of the returns - stock returns are well-known for having fat tails and are difficult to model using a normal distribution (Fama 1965). In fact, modeling a 1% or a 5% quantile of daily returns is a better way to understand and predict what happens on the worst trading days and to give a clearer picture of what might happen during a downturn. Indeed, finance theory suggests that a primary reason why the S&P 500, which is a market-capitalization weighted index composed of the 500-largest publicly traded companies in the United States, has earned a 6.8% inflation-adjusted pre-tax return with dividend reinvestment from January 1871 through April 2020 (PK 2019) is because of the risk of a significant downturn. Kerry Pechter at Forbes describes it as a premium for the fact that “stocks are riskier” and “more prone to price fluctuations in the short run” compared to lower risk investments (Pechter 2020). A portfolio manager must indeed consider the long-run picture; a small difference in the annual rate of return can make an enormous difference in the ending value of investments. However, focusing entirely on long-run value generation is not the only consideration a prudent manager ought to make. While forecasting stock returns in the long-run is challenging, the performance of indices such as the S&P 500, despite seemingly existential threats such as the World Wars and the Great Depression, does give some confidence to investors who try to focus on long-run value generation. Ignoring the short-run reminds one of John Maynard Keynes’ famous maxim that the “long run is a misleading guide to current affairs” because “in the long run we are all dead” (Keynes 1923), and moreover, the short-run impact of a strategy is often more difficult to understand than the long-run results, and potentially more precarious. An investment manager using financial leverage to magnify returns (positive or negative) could be left in dire straits if their investments fell rapidly, despite a sound long-run strategy.

While there are other ways to understand and measure downside risk, a commonly accepted method is using value-at-risk (VaR). The metric is understood as follows: a one-day 1% VaR of -10 million dollars for a portfolio means that the portfolio will lose at least 10 million of its value on the 1% worst trading days. A major advantage of VaR is that it distills a distribution of returns into one number. As such, VaR is often used in stress testing by regulatory agencies in the United States, the United Kingdom, and Europe (Holton 2014).

A popular approach to modeling VaR called RiskMetrics (Longerstaey and Spencer 1996) was introduced by J.P. Morgan in 1994 and re-relased in 1996. The model assumed that a “portfolio or any asset’s returns follow a normal distribution over time” and used this along with the “variance-covariance method” to calculate VaR (Investopedia 2019). While this was certainly a step forward at the time, perhaps the model’s greatest downfall is the pretense of knowledge that modeling the distribution of returns in entirety is possible. The elegant simplicity of using a normal distribution is appealing - only having to estimate the mean and the variance to get a universal picture of returns is certainly appealing, and perhaps necessary in a time of comparatively limited computing power.

Having said that, modeling the big picture while making clear assumptions about the nature of returns has its’ perks, and is perhaps adventagous over altenatives for modeling VaR. Indeed, many of the approaches for modeling VaR rely on a semiparametric or a nonparametric historical simulation (Richardson, Boudoukh, and Whitelaw 2005). According to Robert Engle and Simone Manganelli in a 2004 paper, these methods are usually chosen for “empirical justifications rather than on sound statistical theory” (Engle and Manganelli 2004). They propose a framework called CAViaR that directly forecasts the VaR quantile using a conditional autoregressive quantile specification. This approach builds upon the statistical literature that extends linear quantile models to settings amenable to financial modeling, such as with heteroskedastic and nonstationary error distributions (Portnoy 1991).

Methods Used

The appeal of this model is that it combines the crisp statistical assumptions with the flexibility required to model financial returns. However, the model still runs into issues when a training sample is totally unrepresentative of the testing period - a common problem in statistical analysis. Initial motivations for this paper involved analyzing two stocks - Amazon (ticker: AMZN) and Proctor & Gamble (ticker: PG) and their performance during the Great Recession (specifically, the last 200 trading days of 2008). A relevant question of a financial institution would understandably be how their risk model performed during 2008, a highly volatile period which was driven by the “most severe financial crisis since the Great Depression”, according to Gary Becker (Becker 2008), a Nobel-prize winning economist. Interestingly, the univariate CAViaR forecast for Amazon was fairly accurate whereas the forecast for PG was not. One reason for this could be the fact that a stock like Amazon was highly volatile during the training sample, which included return data starting from the second quarter of 2004, but PG was fairly stable. How would it be possible for a univariate model such as CAViaR, that does not explicitly account for other factors, to forecast well? What if a volatile stock such as AMZN was included into the forecast for PG - would it improve the prediction?

package ‘data.table’ was built under R version 3.5.2data.table 1.12.2 using 1 threads (see ?getDTthreads).  Latest news: r-datatable.com
package ‘quantmod’ was built under R version 3.5.2Loading required package: xts
Loading required package: zoo
package ‘zoo’ was built under R version 3.5.2
Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric


Attaching package: ‘xts’

The following objects are masked from ‘package:data.table’:

    first, last

Loading required package: TTR
Version 0.4-0 included new data defaults. See ?getSymbols.
package ‘ggplot2’ was built under R version 3.5.2package ‘tseries’ was built under R version 3.5.2
    ‘tseries’ version: 0.10-47

    ‘tseries’ is a package for time series analysis and computational finance.

    See ‘library(help="tseries")’ for details.

package ‘dplyr’ was built under R version 3.5.2
Attaching package: ‘dplyr’

The following objects are masked from ‘package:xts’:

    first, last

The following objects are masked from ‘package:data.table’:

    between, first, last

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union

package ‘kableExtra’ was built under R version 3.5.2
Attaching package: ‘kableExtra’

The following object is masked from ‘package:dplyr’:

    group_rows

package ‘quantreg’ was built under R version 3.5.2Loading required package: SparseM

Attaching package: ‘SparseM’

The following object is masked from ‘package:base’:

    backsolve


Attaching package: ‘MTS’

The following object is masked from ‘package:TTR’:

    VMA

package ‘plot3D’ was built under R version 3.5.2package ‘citr’ was built under R version 3.5.2Loading required package: Rcpp
package ‘Rcpp’ was built under R version 3.5.2The working directory was changed to /Users/stevenmoen/Documents/GitHub/CAViaR inside a notebook chunk. The working directory will be reset when the chunk is finished running. Use the knitr root.dir option in the setup chunk to change the working directory for notebook chunks.

Accuracy of VaR Forecast for PG Over Last 200 Trading Days in 2008

	AMZN	PG
VaR Break Rate	0.025	0.055
Theoretical VaR	0.010	0.010
Note:
Tested Using the Symmetric Absolute Value Model

Thus, the idea of combining stocks into a multivariate setting to capture correlations and better forecast risk was formed. A natural choice appeared to be the diffusion index model, originally developed by Stock and Watson for predicting conditional means (Stock and Watson 2002b, 2002a). The model for forecasting the conditional mean is specified below.

Diffusion Index Model

A useful means of predicting stock movements in the future is the Stock and Watson diffusion index. The model is outlined below, which is adapted from Multivariate Time Series Analysis With R and Financial Applications by Ruey S. Tsay (Tsay 2014).

There are two relevant equations, \(\boldsymbol{z_t} = \boldsymbol{Lf_t} + \boldsymbol{\epsilon_t}\) and \(y_{t+h} = \boldsymbol{\beta^\prime f_t} + \boldsymbol{e_{t+h}}\).

In the first equation, \(\boldsymbol{z_t} = (z_{1t}, ...., z_{kt})^\prime\) is an observed time series with mean 0, \(\boldsymbol{f_t}\) is an m-dimensional vector of common factors with mean 0 and identity covariance matrix, \(\boldsymbol{L}\) is a \(k \times m\) loading matrix, and \(\boldsymbol{\epsilon_t}\) is an independent and identically distributed (i.i.d.) sequence of random vectors with mean 0 and covariance matrix \(\boldsymbol{\Sigma_e}\).

In the second equation, which represents the h-step ahead prediction based on \(\boldsymbol{f_t}\), \(y_t\) is the scalar time series of interest, \(h\) is the forecast horizon, \(\boldsymbol{\beta}\) represents the vector of coefficients, and \(e_t\) is a sequence of uncorrelated random variables with mean 0 and constant variance.

To model the data, principal component analysis is performed on the covariates described below to obtain an estimate of \(\boldsymbol{f_t}\). When modeling the conditional mean, the \(\boldsymbol{\beta}\) coefficients are estimated using ordinary least squares, however in the specification below they are not. A specific formulation mentioned in the textbook is as follows, where the individuals diffusion indices are given by \(f_it\), and the goal is a one-step ahead prediction of \(y_t\):

\[ y_{t+1} = \beta_0 + \sum_{i=1}^m \beta_i f_{it} + e_t. \]

Univariate CAViaR Model Specifications

However, work needed to be done to align the diffusion index model with the CAViaR model, which is defined below. The following variables are required for use in the CAViaR model. For ease of notation, these are sourced directly from the Engle and Manganelli 2004 CAViaR paper (Engle and Manganelli 2004), with some added description:

• \((y_t)_{t=1}^T\) is a “vector of portfolio returns”
• \(\theta\) is the “probability associated with VaR” (a 5% VaR would mean \(\theta = 0.05\))
• \(\boldsymbol{x_t}\) is a “vector of time \(t\) observable variables”
• \(f_t(\boldsymbol{\beta}) \equiv f_t(\boldsymbol{x_{t-1}, \boldsymbol{\beta_\theta}})\) is the “time \(t \theta\) quantile of the distribution of portfolio returns formed at time \(t-1\)”

The authors then describe a “generic CAViaR specification” as follows:

\[ f_t(\boldsymbol{\beta}) = \beta_0 + \sum_{i=1}^q \beta_i f_{t-1}(\boldsymbol{\beta}) + \sum_{j=1}^r \beta_j l(\boldsymbol{x_{t-j}}) \]

What is interesting about the general setup is that there are two main components to the model - lagged observed variables (represented by \(l\)) and lagged values of unknown parameters, which in the specification below is used as moving average terms. As such, it is reasonable to generalize the specifications below as nonlinear ARMA models where \(y_{t-1}\) terms refer to previous returns, whereas \(f_{t-1}(\beta_1)\) terms refer to previous predictions.

Adaptive CAViaR Model

Consider the following model:

\[ f_t(\beta_1) = f_{t-1}(\beta_1) + \beta_1\left[\left(1+ \exp(G[y_{t-1} - f_{t-1}(\beta_1)]) \right)^{-1} - \theta \right] \]

Following Engle and Manganelli’s 2004 paper, we choose \(G = 10\), so that is what is used in the results section of this paper. The authors state the reason for the seemingly arbitrary choice is that while “the parameter G itself could be estimated; however, this would go against the spirit of this model, which is simplicity”. Previous sensitivity analysis showed that running the adaptive model with \(G = 5\) did not materially affect the VaR predictions - the accuracy was not changed. While this model is nonlinear in G and total scale invariance in \(G\) would be surprising given the nonlinear relationship, the fact that the other fitted parameters likely adjusted is not surprising.

Symmetric Absolute Value CAViaR Model

Below is the symmetric absolute value CAViaR model:

\[ f_t(\boldsymbol{\beta}) = \beta_1 + \beta_2f_{t-1}(\boldsymbol{\beta}) + \beta_3|y_{t-1}|. \]

Asymmetric Slope CAViaR Model

Below is the asymmetric slope CAViaR model:

\[ f_t(\boldsymbol{\beta}) = \beta_1 + \beta_2f_{t-1}(\boldsymbol{\beta}) + \beta_3(y_{t-1})^+ + \beta_4(y_{t-1})^-. \]

Indirect GARCH (1,1) CAViaR Model

Below is the Indirect GARCH (1,1) model:

\[ f_t(\boldsymbol{\beta}) = (\beta_1 + \beta_2f_{t-1}^2(\boldsymbol{\beta}) + \beta_3y_{t-1}^2)^{1/2}. \]

Multivariate CAViaR Model Specifications

The multivariate CAViaR model takes inspiration from the models described above in several specifications, as mentioned in the original specifications. The general model form looks like the specification below:

\[ f_t(\boldsymbol{\beta}) = \beta_0 +\sum_{i=1}^p \beta_i y_{t-i} + \sum_{j=1}^m \beta_{j+p} f_{j,t-1} + e_t. \]

As with the univariate CAViaR model, the object of interest is a \(\theta\) percentile return and the model is fit iteratively to minimize the loss function on the training data. However, there are some notable differences between the univariate model and the multivariate model. First, there are no moving average terms (lagged error terms) - the reasoning for this is because this model aims for a clear economic interpretation, and crisp interpretations of MA models are harder to create. Also, moving average models require recursive estimation since error terms are not observed, and so developing a method to work with these errors in a robust regression framework is challenging.

Second, in some of the specifications below, there are lagged return variables. This is similar to the univariate CAViaR specification, though there is often more than 1 lag as in the univariate model - there are \(p\) lags in the dataset. Third, in all of the specifications below, there are \(m\) diffusion indices used in each model lagged by one time step to avoid look-ahead bias.

Multivariate CAViaR: No Lags Model

\[ f_t(\boldsymbol{\beta}) = \beta_0 +\sum_{j=1}^m \beta_{j} f_{j,t-1} + e_t \]

Multivariate CAViaR with Autoregressive Terms Added

\[ f_t(\boldsymbol{\beta}) = \beta_0 +\sum_{i=1}^p \beta_i y_{t-i} + \sum_{j=1}^m \beta_{j+p} f_{j,t-1} + e_t \]

Multivariate CAViaR with Symmetric Absolute Value Autoregressive Terms Added

\[ f_t(\boldsymbol{\beta}) = \beta_0 +\sum_{i=1}^p \beta_i |y_{t-i}| + \sum_{j=1}^m \beta_{j+p} f_{j,t-1} + e_t \]

Multivariate CAViaR with Asymmetric Slope Autoregressive Terms Added

\[ f_t(\boldsymbol{\beta}) = \beta_0 +\sum_{i=1}^p \beta_i (y_{t-i})_+ + \sum_{j=p+1}^{2p} \beta_i (y_{t-i})_- + \sum_{k=1}^m \beta_{k+2p} f_{k,t-1} + e_t \]

Fitting the Models

To fit the models, an optimal value of \(m\) diffusion indices and \(p\) autoregressive terms are added (or \(2p\) in the case of the asymmetric slope model). The optimal values of these parameters are determined using a validation dataset. In all of the runs below, there are a total of 5 years of trading days, or about 1,260 days assuming 252 trading days a year. The adjusted closing prices are logged and differenced, shortening the dataset by one. After doing this, the last 250 data points are reserved as test data, and the 250 data points before that are used as a validation set. Measured by the loss function written out below, the values of \(p\) and \(m\) that minimize losses are chosen and the optimal model is refit over both the training and the validation data combined and then evaluated on the test data. Note that there is an optimal model is chosen for each of the four multivariate CAViaR specifications described above, so there are 4 optimal sets of \(p\) and \(m\) chosen for each set of model. Thus, there are 8 models compared on the test data - 4 univariate CAViaR models and 4 multivariate CAViaR models.

From the CAViaR paper, the \(\theta\)th regression quantile is defined as any \(\boldsymbol{\hat{\beta}}\) that solves the following loss function:

\[ \overset{argmin}\beta \frac{1}{T}\sum_{t=1}^T [\theta - I(y_t < f_t(\boldsymbol{\beta}))][y_t - f_t(\boldsymbol{\beta})] \]

Theoretical Guarantees of Consistency and Asymptotic Normality

Part of the reason for working with the CAViaR and diffusion index is their strong theoretical guarantees about consistency and asymptotically. Indeed, following the results in Engle and Manganelli (Engle and Manganelli 2004), there are 8 conditions required for consistency of the \(\beta\) estimate and 4 required for asymptotic normality. The paper states that the model specified by:

\[ y_t = f(y_{t-1}, \boldsymbol{x}_{t-1},..., y_1, \boldsymbol{x}_1; \boldsymbol{\beta}^0) + \epsilon_{t \theta} [Quant_\theta (\epsilon_{t \theta} | \Omega_t) = 0] \] \[ \equiv f_t(\boldsymbol{\beta^0}) + \epsilon_{t \theta}, t = 1,...,T \]

“where \(f_1(\boldsymbol{\beta^0})\) is some given initial condition, \(\boldsymbol{x_t}\) is a vector of exogenous of predetermined variables, \(\boldsymbol{\beta^0} \in \mathbb{R}^p\) is the vector of true unknown parameters that need to be estimated, and \(\Omega_t = [y_{t-1}, \boldsymbol{x}_{t-1},...,y_1, \boldsymbol{x}_1; f_1 (\boldsymbol{\beta}^0)]\) is the information set available at time \(t\)”, and \(\boldsymbol{\hat{\beta}}\) is the paramter vector that minimizes the loss function specified above. According to theorems in the paper, they state that under favorable conditions, \(\boldsymbol{\hat{\beta}}\) is consistent and asymptotically normal.

Consistency

Per Engle and Manganelli, under the model specified above and using 8 assumptions given below, \(\boldsymbol{\hat{\beta}} \overset{p}\rightarrow \boldsymbol{\beta^0}\) where \(\boldsymbol{\hat{\beta}}\) is the paramter vector that minimizes the loss function specified above. There are 8 assumptions listed in the paper; most seem fairly standard.

1. “\((\Omega, F, P)\) is a complete probability space, and {\(\epsilon_{t \theta}\), \(\boldsymbol{x}_t\)}, t = 1,2,…. are random vectors on this space”
2. “The function \(f_t(\boldsymbol{\beta}): \mathbb{R}^{k_t} \times B \rightarrow \mathbb{R}\) is such that for each \(\boldsymbol{\beta} \in B\), a compact subset of \(\mathbb{R}^p, f_t(\boldsymbol{\beta})\) is measurable with respect to the information set \(\Omega_t\) and \(f_t(\cdot)\) is continuous in \(B\), \(t = 1,2,....,\) for a given choice of explanatory variables {\(y_{t-1}, \boldsymbol{x}_{t-1}, ..., y_{1}, \boldsymbol{x}_{1}\)}.”
3. “Conditional on all of the past information \(\Omega_t\), the error terms \(\epsilon_{t \theta}\) form a stationary process, with continuous conditional density \(h_t(\epsilon | \Omega_t)\).”
4. “There exists \(h > 0\) such that for all \(t, h_t(0 | \Omega_t) \geq h\).”
5. “|\(f_t(\boldsymbol{\beta})\)| < \(K(\Omega_t)\) for each \(\boldsymbol{\beta} \in B\) and for all \(t\), where \(K(\Omega_t)\) is some (possibly) stochastic function of variables that belong to the information set, such that \(\mathbb{E}(|K(\Omega_t)|) \leq K_0 < \infty\), for some constant \(K_0\)”
6. “\(\mathbb{E}[|\epsilon_{t\theta}|] < \infty\) for all \(t\)”
7. “{\([\theta - I(y_t < f_t(\boldsymbol{\beta}))][(y_t - f_t(\boldsymbol{\beta}))]\)} obeys the uniform law of large numbers”
8. “For every \(\xi > 0\), there exits a \(\tau > 0\) such that if ||\(\boldsymbol{\beta} = \boldsymbol{\beta}^0\)|| \(\geq \xi\), then \(\lim \inf_{T \rightarrow \infty} T^{-1} \Sigma P[|f_t(\boldsymbol{\beta}) - f_t(\boldsymbol{\beta^0})| > \tau] >0\)”

When analyzing real data, it’s hard to verify any assumptions exactly, but one that is most controversial might be the third assumption - indeed, it seems highly unlikely that given all the past information, there would be a stationary process.

Asymptotic Normality

Also per Engle and Manganelli, under the same assumptions required for consistency as well as the assumptions below, there is a guarantee of asymptotic normality:

\[ \sqrt{T} \boldsymbol{A}_T^{-1/2}\boldsymbol{D}_T (\boldsymbol{\hat{\beta}} - \boldsymbol{\beta}^0) \overset{d}\rightarrow \mathcal{N}(0, \boldsymbol{I}) \]

where

\[ \boldsymbol{A}_T \equiv \mathbb{E} \left[T^{-1} \theta (1-\theta) \sum_{t=1}^T \nabla' f_t(\boldsymbol{\beta}^0) \nabla f_t(\boldsymbol{\beta}^0) \right] \]

and

\[ \boldsymbol{D}_T \equiv \mathbb{E} \left[T^{-1} \sum_{t=1}^T h_t(0|\Omega_t)\nabla' f_t(\boldsymbol{\beta}^0) \nabla f_t(\boldsymbol{\beta}^0) \right] \]

There are 4 assumptions listed in the paper required for asymptotic normality to hold. As with the assumptions required for consistency, these seem fairly standard as well:

1. “\(f_t (\boldsymbol{\beta})\) is differentiable in \(B\) and for all \(\boldsymbol{\beta}\) and \(\boldsymbol{\gamma}\) in a neighborhood \(\nu_0\) of \(\boldsymbol{\beta^0}\), such that \(||\boldsymbol{\beta} - \boldsymbol{\gamma}|| \leq d\) for \(d\) sufficiently small and for all \(t\):”

1. “\(||\nabla f_t (\boldsymbol{\beta})|| \leq F(\Omega_t)\), where \(F(\Omega_t)\) is some (possibly) stochastic funciton of variables that belong to the information set and \(\mathbb{E}(F(\Omega_t)^3) \leq F_0 < \infty\), for some constant \(F_0\).”
2. “\(||\nabla f_t (\boldsymbol{\beta})- \nabla f_t (\boldsymbol{\gamma})|| \leq M(\Omega_t, \boldsymbol{\beta}, \boldsymbol{\gamma}) = \mathcal{O}(||\boldsymbol{\beta} - \boldsymbol{\gamma}||)\), where \(M(\Omega_t, \boldsymbol{\beta}, \boldsymbol{\gamma})\) is some function such that \(\mathbb{E}[M(\Omega_t, \boldsymbol{\beta}, \boldsymbol{\gamma})]^2 \leq M_0||\boldsymbol{\beta} - \boldsymbol{\gamma}|| < \infty\) and \(\mathbb{E}[M(\Omega_t, \boldsymbol{\beta}, \boldsymbol{\gamma})]F(\Omega_t)] \leq M_1|\boldsymbol{\beta} - \boldsymbol{\gamma}|| < \infty\) for some constants \(M_0\) and \(M_1\).” 2a. “\(h(\epsilon|\Omega_t) \leq N < \infty \text{ } \forall t\), for some constant \(N\).”
3. “\(h(\epsilon|\Omega_t)\) satisfies the Lipschitz condition \(|h_t(\lambda_1 | \Omega_t) - h_t(\lambda_2 | \Omega_t)| \leq L|\lambda_1 - \lambda_2|\) for some constant \(L < \infty \text{ } \forall t\).”

3. “The matrices \(\boldsymbol{A}_T \equiv \mathbb{E} \left[T^{-1} \theta (1-\theta) \sum_{t=1}^T \nabla' f_t(\boldsymbol{\beta}^0) \nabla \times f_t(\boldsymbol{\beta}^0) \right]\) and \(\boldsymbol{D}_T \equiv \mathbb{E} \left[T^{-1} \sum_{t=1}^T h_t(0|\Omega_t)\nabla' f_t(\boldsymbol{\beta}^0) \times \nabla f_t(\boldsymbol{\beta}^0) \right]\) have the smalles eigenvalues bounded below by a positive constant \(T\) for sufficiently large.”
4. "The sequence {\(T^{-1/2} \sum_{t=1}^T[\theta - I(y_t < f_t(\boldsymbol{\beta}^0))] \nabla'f_t(\boldsymbol{\beta}^0)\)} obeys the central limit theorem.

As with the consistency conditions, these seem reasonable enough - the data considered in this analysis seems well-behaved enough such that these conditions are satisfied.

Data Used

The response variable used in this analysis is SPY, which is an exchange-traded fund that aims to track the performance of the S&P 500, which is discussed above. It is broadly used as a bellwether of the U.S. economy, and has the advantage of avoiding survivorship bias - while an individual stock might go bankrupt or merge with another, it is reasonable to assume that these issues do not apply with an ETF.

Following this logic, there are several classes of response variables used in this analysis. The first group is a set of U.S. sector ETFs obtained from Seeking Alpha (NA 2020). As with the response variable, these ETFs were publicly traded throughout the Great Recession of 2008.

1. Utilities (XLU)
2. Consumer Staples (XLP)
3. Healthcare (XLV)
4. Technology (XLK)
5. Consumer Discretionary (XLY)
6. Industrial (XLI)
7. Financial Services (XLF)
8. Basic Materials (XLB)
9. Energy (XLE)

The second group is Global Sector ETFs, also from Seeking Alpha (NA 2020). The rationale for including these is that perhaps some global exposure is useful in understanding the broader market.

1. Utilities (JXI)
2. Consumer Staples (KXI)
3. Healthcare (IXJ)
4. Telecommunications (IXP)
5. Technology (IXN)
6. Consumer Discretionary (RXI)
7. Industrial (EXI)
8. Financial Services (IXG)
9. Basic Materials (MXI)
10. Energy (IXC)

The third group is bond ETFs. Like the previous two groups, these ETFs potentially contain forward-looking information about the stock market. These ETFs were chosen because they were the first fixed-income ETFs available in the United States, and had enough history for this paper (NA 2017).

1. iShares 1-3 Year Treasury Bond Fund (SHY)
2. iShares 7-10 Year Treasury Bond Fund (IEF)
3. iShares 20+ Year Treasury Bond Fund (TLT)
4. iShares iBoxx $ Investment Grade Corporate Bond ETF (LQD)

Lastly, all of the above three groups are run together. One reason for having bond and stocks grouped together is the fact that bonds are somewhat of a substitute for equities, which tend to drop more in a period of crisis (Amadeo 2020). As such, some unexplained movements in the stock price could be picked up by bond movements.

In each run, the explanatory variables are lagged to avoid look-ahead bias. All of the runs analyze the difference of the log of the adjusted closing price. The reason for using the differenced log is that it closely approximates the percentage change of the price for small changes. The reason for using the adjusted closing prices is that an adjusted closing price excludes the effects of “corporate actions such as stock splits, dividends / distributions and rights offerings” (Gant 2019). While dividends are essential to study the long-term performance of a strategy, studying short-term price movements do not require understanding the effects of dividend reinvestment. While there are many candidate ETFs chosen, these were chosen because they all had price history going back through the beginning of 2004.

Results

Big Simulation Function

For the sake of brevity, the results with only U.S. ETFs, global ETFs, or bonds is included in a later results section. The results are similar to the results below. To test how well the models do at different VaR levels, 1%, 5%, and 10% are tested.

2008 Test Period - All ETFs

1% VaR

5% VaR

10% VaR

The univariate CAViaR models significantly outperform the multivariate model, particularly at the 1% level. The extreme behavior towards the end of 2008 proved difficult for the multivariate model to pick up on.

2010 Test Period - All ETFs

1% VaR

5% VaR

10% VaR

The multivariate forecast is largely in line with the univariate CAViaR model. Generally, it seems like the univariate model does a better job tracking the response variable in the case of a large swing because of the moving average component. Also, while the multivariate models had a rate of VaR breaks that was too high for 2008, the rate of VaR breaks was generally too low for the multivariate models in 2010.

2014 Test Period - All ETFs

1% VaR

5% VaR

10% VaR

As with the 2010 results, the multivariate and univariate forecasts are very similar in accuracy for all 3 VaR levels. In terms of the VaR breakage rate for the 1% level, 3 out of the 4 multivariate models were extremely close. There also appears to be less differentiation between the multivariate and the univariate models compared to the 2010 and 2014 results.

2016 Test Period - All ETFs

1% VaR

5% VaR

10% VaR

As with 2010 and 2014, the results between model classes are similar. The multivariate model performs quite well with VaR breaks at the 5% and 10% level, and the models seem to be very closely clustered together.

Conclusions and Future Work

The problem of how to predict a low quantile of a stock’s log return when the training sample is substantially different from the test scenario is an enormously difficult problem. Almost axiomatically, the distribution is nonstationary over time. How is it possible to predict the return of an index like the S&P500 during a period of market turmoil such as the Great Recession? While the univariate CAViaR model performs comparatively well during times of stress, it performs about the same as the multivariate CAViaR model during more benign economic periods.

This conclusion drawn from the above results might support the notion of combining the two models in some sort of a mixture model - aiming to use the basket of ETFs during good times, and use the CAViaR ARMA specification during bad times. The approach of using ETFs allows a prediction based on forward-looking expectations of fundamental factors. Indeed, ETFs are just baskets of individual stocks or bonds, and those securities are (in theory) based on rational expectations about future resources, market conditions, etc - the microfoundations of what drives our economy. The ARMA specification, while practically and statistically sound, is contradicted by economic theory and practice - the weak form of the efficient market hypothesis states that it is impossible to forecast future values of asset prices using past values. But perhaps this view is incomplete.

Any model that attempts to capture relationships in the real world will only work until an omitted variable is found. The elegance of the multivariate CAViaR model is that it provides insight into why a prediction is wrong; the change in the angle between resultant vectors is a sensible measurement of economic changepoints. However, errors in the world are costly, and it is wishful thinking to say that explaining why the error occurred is sufficient.

As such, for future work it is worth exploring the notion of weighting an ARMA-approach more heavily when predictions using fundamentals were too high, then not only would this after-the-fact recognition be achieved, but also a hierarchical model that captures fundamental relationships in the economy and potentially changes our understanding of asset prices in general - a synthesis between Keynes’ animal spirits during a time of severe crisis; where a model cannot explain shifts, and a more rational world that explains other periods. In addition to significant predictive power because of the switching between the two worlds, there is also an elegant explanation; a way to explain changes in the usefulness of the underpinnings in the economy. Because of the flexibility of the model, it is entirely possible that a whole gamut of variables could be tossed in and backtested to when “changepoints” occurred.

Additional future work involves developing theoretical guarantees on the parameters in the multivariate CAViaR model. One advantage of both the diffusion index model and the CAViaR model is that both have theorems about asymptotic normality and consistency.

Additional Results

2008 Test Period

U.S. ETFs

The results for the U.S. ETFs clearly show that the univariate model outperforms the multivariate model during the great recession.

Global ETFs

While the model with the global ETFs as predictors performs better than the model with U.S. ETFs, the univariate CAViaR model outperforms the multivariate model.

Bond ETFs

As with the previous two runs, the univariate model outperforms the multivariate model.

2010 Test Period

U.S. ETFs

The results are much better for the multivariate model here during a more benign period.

Global ETFs

As with the U.S. ETFs, the results are better.

Bond ETFs

As with the previous two runs, the results from the bond ETFs are better than those from 2008.

2014 Test Period

U.S. ETFs

As with the results in 2014, the results from the U.S. ETFs shows a similar quality of predictions.

Global ETFs

The results show that the multivariate predictions are of similar quality to the predictions from the univariate model.

Bond ETFs

As with the other two sets of regressors, the results here show that the multivariate predictions are of similar quality to the predictions from the univariate model.

2016 Test Period

U.S. ETFs

As with the forecast in 2010 and 2014, including the U.S. ETFs as predictors in the multivariate model gives similar results to that of the univariate model.

Global ETFs

The global ETFs as predictors provide solid results as well.

Bond ETFs

As with the U.S. and Global ETFs, the bond ETFs provide good results.

Code Appendix

Literature Cited

Amadeo, Kimberly. 2020. “How Bonds Affect the Stock Market.” The Balance. https://www.thebalance.com/how-bonds-affect-the-stock-market-3305603.

Becker, Gary. 2008. “We’re Not Headed for a Depression.” https://www.wsj.com/articles/SB122333679431409639.

Engle, Robert F, and Simone Manganelli. 2004. “CAViaR.” Journal of Business & Economic Statistics 22 (4). Taylor & Francis: 367–81. https://doi.org/10.1198/073500104000000370.

Fama, Eugene F. 1965. “The Behavior of Stock-Market Prices.” The Journal of Business 38 (1). University of Chicago Press: 34–105. http://www.jstor.org/stable/2350752.

Gant, Akhilesh. 2019. “Adjusted Closing Price Definition.” https://www.investopedia.com/terms/a/adjusted{\_}closing{\_}price.asp.

Holton, Glyn A. 2014. “History of VaR - Value-at-Risk: Theory and Practice.” https://www.value-at-risk.net/history-of-value-at-risk/.

Investopedia. 2019. “What is RiskMetrics in Value at Risk ( VaR )?” https://www.investopedia.com/ask/answers/041615/what-riskmetrics-value-risk-var.asp.

Keynes, John Maynard. 1923. A tract on monetary reform, London: Macmillan; Co., Limited.

Longerstaey, Jacques, and Martin Spencer. 1996. “RiskMetrics - Technical Document.” New York: J.P. Morgan/Reuters. http://www.jpmorgan.com/RiskManagement/RiskMetrics/RiskMetrics.html.

NA. 2017. “iShares Institutional Guide to Bond ETFs.” https://www.complianceweek.com/download?ac=5780.

———. 2020. “Sector ETFs | Seeking Alpha.” https://seekingalpha.com/etfs-and-funds/etf-tables/sectors.

Pechter, Kerry. 2020. “You Need to Understand the ‘Equity Risk Premium’.” https://www.forbes.com/sites/kerrypechter/2020/04/04/erp-zirp-and-when-to-get-back-into-the-market/{\#}36566af6e4d1.

PK. 2019. “S&P 500 Return Calculator, with Dividend Reinvestment.” https://dqydj.com/sp-500-return-calculator/.

Portnoy, Stephen. 1991. “Asymptotic behavior of regression quantiles in non-stationary, dependent cases.” Journal of Multivariate Analysis 38 (1): 100–113. https://doi.org/https://doi.org/10.1016/0047-259X(91)90034-Y.

Richardson, Matthew P., Jacob (Kobi) Boudoukh, and Robert F Whitelaw. 2005. “The Best of Both Worlds: A Hybrid Approach to Calculating Value at Risk.” SSRN Electronic Journal. https://doi.org/10.2139/ssrn.51420.

Stock, James H, and Mark W Watson. 2002a. “Macroeconomic Forecasting Using Diffusion Indexes.” Journal of Business & Economic Statistics 20 (2). Taylor & Francis: 147–62. https://doi.org/10.1198/073500102317351921.

———. 2002b. “Forecasting Using Principal Components from a Large Number of Predictors.” Journal of the American Statistical Association 97 (460). [American Statistical Association, Taylor & Francis, Ltd.]: 1167–79. http://www.jstor.org/stable/3085839.

Tsay, Ruey. 2014. Multivariate Time Series Analysis. John Wiley & Sons, Inc.